Control and Spacing 1 Metacognitive control and the spacing effect

This study investigates whether the use of a spacing strategy absolutely improves final performance, even when the learner had chosen, metacognitively, to mass. After making judgments of learning, adult and child participants chose to mass or space their study of word pairs. However, a third of their choices were dishonored. That is, they were forced to mass after having chosen to space and forced to space after having chosen to mass. Results showed that the spacing effect obtained for both adults and children when choices were honored. However, using a spacing strategy when it was in disagreement with one’s own choice, or forced, did not enhance performance for the adults (Experiment 1). And although it was enhanced for the children (beyond massing strategies), the performance was not as good as that when the spacing decisions were self-chosen (Experiment 2). The data suggest that although spacing is an effective strategy for learning, it is not universal, particularly when the strategy was not chosen by the learner. In short, metacognitive control is often crucial and should be honored. Control and Spacing 2 On the whole, massing has acquired a bad reputation. Synonymous with cramming, the practice has been described as suboptimal, reducing, and even harmful. A number of reviews and meta-analyses on existing data have showcased the drawbacks of massed study, particularly during long-term learning (see Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006, for a review). Conversely, since being first described in 1885 by Ebbinghaus, spacing, or distributing study across time in short interrupted sessions, has been shown to boost learning. Data and theory behind this so-called spacing effect has been so overwhelming, that it was at one point considered a ―law‖--Neisser’s law (taken from Bjork, 1988, p.399): ―You can get a good deal from rehearsal, If it just has the proper dispersal. You would just be an ass, To do it en masse, Your remembering would turn out much worsal.‖ The question addressed in this paper is: Does using a spacing strategy absolutely improve learning, even if that strategy goes against one’s metacognitive choice? Spacing imperfections The spacing effect is one of the most well documented cognitive phenomena in the literature (e.g., Bahrick, 1987; Bjork, 1979; Dempster, 1987; Glenberg, 1979). The effect has also been obtained in children (Cahill & Toppino, 1993; Rea & Modigliani, 1987; Toppino, 1991, 1993; Toppino & DiGeorge, 1984), and is well advocated by researchers and educators alike. Data supporting the spacing effect have also been backed by explanations for why spacing works. The retrieval hypothesis (Glover, 1989), which suggests that people take part in a more effortful retrieval process during spaced study than during massed study, has been proposed as one mechanism driving the effect. Effortful processing may strengthen or create ―retrieval routes‖ to the correct Control and Spacing 3 representation of the target in memory (e.g. Bjork, 1975). Unfortunately, the extra effort that is necessary during spaced practice might also turn people away from spacing, and towards massing strategies. Studies have, in fact, shown that during spaced study (more than massed study), acquisition is not only slower, but confidence is lower also (Bahrick et al., 1993; Zechmeister & Shaughnessy, 1980). In addition to its potential unpopularity, there is also a documented set of circumstances where spacing is not always the better strategy. During one particular type of study using expanding retrieval practice, one first retrieves the to-be-remembered item at short lags—massed—and then with successive study trials, waits longer before trying to retrieve the desired item from memory—spaced (e.g. Landauer & Bjork, 1978). In Landauer and Bjork’s study (1978), for instance, participants were presented with massed, spaced, and expanding cue-target pairs, interwoven into one long list. Each name was presented 4 times, the first time being a presentation trial, and the final three being retrieval practice trials (i.e. cue-only) without feedback. After the entire list had been presented, a cued-recall test was given. The results showed that final performance was better for spaced than for massed items—in line with the usual spacing effect—but better yet for the expanding items, where study consisted partly of massing. The authors theorized that spacing is a good strategy, but only if retrieval is successful (also see Bahrick, 1987; Melton, 1970; Rose, 1984; & Underwood, 1961, for similar theories). In the expanding condition, items were spaced far enough apart that people benefited from Control and Spacing 4 effortful retrieval, but not so far apart that retrieval was unlikely to be successful (i.e. during early learning). 1 Other human and model data have supported the general notion of the success in an expanding practice paradigm. For instance, more recently, Pavlik and Anderson (2005) have shown, using the ACT-R model, that spacing is beneficial, but that the effect is strengthened as more practice is accumulated. These findings suggest that interrupting study and using a spaced strategy would be best only when learning has reached some criterion level in the first place. On the other hand, when the item still feels novel or is situated in a fragile state, it would behoove the learner to mass study, allocating current and continuous time to help progress the item to a more secured position in memory. Indeed, recent findings have shown that people will choose to mass their study when encoding has not fully occurred (Toppino, Cohen, Davis, & Moors, in press). This question of how people choose to schedule their study is next addressed. Metacognitive spacing Metacognitive control can be defined as the process of using one’s own judgments to guide behavior. For example, a number of studies have tested the relation between one’s judgments and the amount of time one allocates to study (see Son & Kornell, 2008, for a review). In general, data have shown that people choose to allocate the most study time to relatively difficult items, so long as learning is not hopeless and enough time is available (Kornell & Metcalfe, 2006; Metcalfe & Kornell, 2005). The issue of metacognitive control of spacing has been less investigated (Benjamin & Bird, 2006; Son, 2004). In one study (Son, 2004), people were asked to make metacognitive 1 Recently, not all of the retrieval practice results have been consistent (see, for instance, Karpicke & Roediger, 2007, and Balota, Duchek, Sergent-Marshall, & Roediger, 2006). Control and Spacing 5 judgments for cue-target pairs. Then they had to decide whether they wanted to mass or space the item’s subsequent study session. If they chose to mass, then the pair would be shown again immediately in succession; If they chose to space, then the pair would not be shown again until after the entire list had been presented. The results showed that when given the choice, adults followed what seems effective. That is, they spent more time spacing than massing, and their metacognitions guided their decisions in accordance to the expanding practice schedule: They spaced items that were judged easy and massed items that were more difficult. However, these findings were contradicted in another study (Benjamin & Bird, 2006)--people chose to space the more difficult items and mass the easier items. There were, though 2 main differences between the two studies. In Son's (2004) study, people were given the option of not studying, that is, they were given a "done" button; Benjamin and Bird's (2006) methods did not have this option--all of the items had to be re-studied, in either a massed or a spaced manner. In addition, in Son's study, any number of items could be massed or spaced. In other words, if an individual wanted to mass every single item, they could choose to do so. In Benjamin and Bird's study, half of the items had to be massed, and half spaced. Recent findings have since rectified the inconsistency in people's spacing choices as driven by judged difficulty. Namely, people will choose to mass their study when encoding has been insufficient; people will tend to space their study when encoding is sufficient (Toppino, et al., in press). Given the previous data and this new finding, a critical implication arises: one should be aware of how "well encoded" items are before choosing to mass or space their study. If one were to follow a simple rule, perhaps it would be the following: It's Control and Spacing 6 advantageous to space my study, but only when I feel that I've encoded the item sufficiently. If it's not sufficiently encoded, then I should continue to study the item now. Logically speaking then, the spacing effect should not be universal, that is, the effect should disappear if the situation made it so that sufficient encoding could not be achieved. An example of this would be when one learns items that are extremely difficult, and the mere task of reading them might take significant amounts of effort. This implication is further discussed in the final section. How do children choose to space out their study? One study investigated spacing choices in children empirically (Son, 2005)-Children in first grade were tested on a similar paradigm (using age-appropriate materials) as the one used with adults in Son (2004) where massing and spacing choices needed to be made. Results showed both a lack of metacognitive strategy as well as a larger inclination to mass than to space. Even for the items that were judged to be very easy, children preferred a massing strategy, going against what is thought to be best according to the expanding paradigm 2 . Such results may be due to the hypothesis that children are likely to be less aware of how well encoded any to-be-learned materials are, and on top of that, have experienced less various study strategies than adults have, including the experience of benefiting from spacing. Given the data regarding metacognitive control of spacing (in particular the lack of it in children) and in light of the overwhelming evidence for the spacing effect, the 2 Different from the paradigm used with adults in Son (2004), for the children there was no ―done‖ option because in a previous study, when the done option was available, children tended to only choose that option. That is, they simply did not want to study further. Thus, although it would be good to compare both children and adults using a similar paradigm, given that children’s motivations strategies may be different from adults, it is almost impossible to set up a perfect comparison, both from the past studies, and in this paper’s study. Control and Spacing 7 following question was asked: Would it be beneficial to impose a spacing strategy on the learner, even while defying one’s metacognitive choice? The current study tested this question by having people learn synonym pairs, make metacognitive judgments, and then decide whether they wanted to mass or space their study for each item. The computer then gave them massed or spaced study sessions, either in agreement with their choice, or against their choice. The critical question was to see whether performance would be best for those items that were given spaced study (even though they had been chosen for massed study). Or, could the spacing effect be eliminated? Experiment 1A As described above, one set of data has shown that adults use a systematic strategy for spacing (Son, 2004). Although they choose to mass relatively difficult items, they tend to space the easier items, which presumably get an additional boost from the spacing effect. Is it reasonable to think that if forced to space their study, including items that were selected for massed practice, performance would increase yet further? One prediction could be that the spacing effect is robust enough that regardless of metacognitive control, and regardless of the hypothesis that massing might benefit under some situations, having people space their study would heighten learning. On the other hand, adults are systematic in their choices, and may make those choices based on real benefits, which might be evidence enough to reveal that metacognitive control of spacing works and shouldn’t be disturbed. Method Control and Spacing 8 Participants. Thirty-one introductory psychology students participated for course credit. Participants were treated in accordance with the ethical standards of the American Psychological Association. Materials. The stimuli were 60 synonym pairs taken from a list of vocabulary words from the Graduate Record Examination (e.g. hirsute-hairy), randomly selected by the computer program for each participant from a pool of about 100 word pairs. Through random selection, it was hoped that a range of difficulty would be presented to each participant. The level of difficulty was assessed by each individual, using a JOL procedure typically used in the field. Procedure. The general procedure came from Son's methods (2004). Participants were tested on macintosh computers. Upon beginning the experiment, the instructions for a practice session appeared on the screen, as follows: "Welcome to the Experiment! In this experiment, you will be presented with a series of 60 words and their synonyms to study. Please stay as attentive as you can. For each pair, you will be asked to make a rating of how confident you are that you will be able to type in the synonym when given only the word on a later memory test. You will be making your response on a slider, ranging from 0 to 100. If you are confident that you will not know the synonym given the word then move the slider to the far left where it is labeled "Don't Know". If you are completely sure that you will be able to type in the synonym given the word, then move the slider to the far right, where it is labeled "Know". If your confidence is somewhere in between, then move the slider accordingly. Your response is called a Judgment of Learning, or JOL. Once you have moved the slider, click on "Submit JOL". You should submit your JOLs as quickly and as accurately as possible. If you take too long, then the Control and Spacing 9 computer will remind you to "Hurry!" When you move the slider to enter your response, a message will appear informing you of your JOL 3 . Your JOL will not be recorded, however, until you have clicked on a button that is marked "Submit JOL". Try a couple of practice trials first." Once participants clicked on a "ready" button, they were given a set of five word-synonym pairs to study and make JOLs. Each of the pairs was presented for 1 second each, and then the slider appeared labeled from 0-100. Once the JOL practice session ended, another set of instructions was displayed, this time explaining the massing and spacing strategies, as follows: "Very good! From here on out, the value of your JOLs will not be shown to you. After you have made your JOL using the slider for each word pair, you will then have another chance to study that word. You will have 3 choices. You can either study the word again immediately or you can wait to study the word again later on in the list. You may also choose not to study that word any longer. Three buttons will appear on the screen like this: [the buttons were labeled "study now", "study later", and "done"]. If you want to study again immediately, click on "Study Now". Once you do, the same word and its synonym will appear again on the screen immediately--most of the time--For a select very few of those trials, they will appear again later. If you want to study again later, click on "Study Later". Once you do, the list will move on to the next word, while that word is saved for later study (Again, for a select very few, that word will not be saved for later and instead, will be shown again immediately). If you no longer want to study a word, click on "Done". These items will not be shown again at all. Although there will be a few trials during which you will get 3 This was so that the participants could get an idea of the numeric value of their JOL in terms of the position on the slider. During the actual experiment, these values were no longer shown. Control and Spacing 10 the opposite of what you selected, please be as sincere as possible in your re-study choices. When you are ready to begin study, click below. As soon as you click, the first word will appear." If people chose to mass, the same pair would be presented again immediately for 3 seconds. If they chose to space, then the pair would not be re-presented until after having studied the entire list (again for 3 seconds). However, as was explained in the instructions, their choices were honored only some part of the time. Specifically, their choices were honored two-thirds of the time. On the remaining third of the trials, their choices were dishonored. That is, the computer forced participants to space when they chose to mass, and mass when they chose to space (see Kornell & Metcalfe, 2006, for the original use of the honor/dishonor paradigm in relation to study time allocation). All of the done items were honored, that is, items for which Done was selected were dropped from any further study. Once all of the pairs had been presented a second time (if chosen for re-study), then the next phase began where people had to calculate 2-digit by 2-digit multiplication problems for 5 minutes on paper provided next to the computer. Participants were told to work on the math problems until they heard the computer beep. Along with the beep, instructions for the final test were presented on the screen, as follows: "You are now ready to take the synonyms test. Each word will be presented to you on the screen. You task is to type in the synonym. Try to type in the EXACT synonym that you had studied earlier in the list. If you do not remember the correct answer, you can leave the item blank. Press click on ―Begin Test‖ when you are ready." Each word was displayed randomly for each subject. All of the data, including JOL, Control and Spacing 11 strategy choice, whether the choice was honored or dishonored, and final test performance, were all recorded by the computer. Results. The JOL data were analyzed using normalized JOL scores, divided into 6 levels (vincentized into the top 1/6, next 1/6, etc.) for each participant. That is, for each individual participant, the JOLs were divided into sixths—depending on their mean JOLs and variations—and then re-categorized from 1(low JOLs) to 6 (highest JOLs). Normalized JOLs were used specifically because of the variance in the use of the scale. For instance, some participants may have used the entire scale (from 0-100), while other might have used only a portion of the scale (0-50). Then, for the former subject their highest JOL category might include those raw JOLs in the 80s and 90s; for the latter participant, the highest JOL category might include those raw JOLs in the 40s. Thus, for the final performance results, ANOVAs were conducted with JOL category level (1-6), honor/dishonor, and massing/spacing choice, as the main variables. JOL data. Before showing the main results, one can ask how JOLs related to both final test performance as well as study choice in general. Although a clean correlation between judgments and test performance cannot be achieved because of the additional study session in between, the rank-order Gamma (G) correlation (see Nelson, 1984) between judgments and final test performance was very high (M = 0.43), suggesting that items that were judged easier were, in fact, remembered better. A one-sample t-test showed the correlation to be significantly different from zero, t(29) = 11.84, SE = .04. How did JOL affect study choice? The top panel of Figure 1 displays adult choices at each level of normalized JOL. As can be seen, as JOL increased, people's Control and Spacing 12 selections for spaced and done items increased while the number of massed items decreased. This trend replicates the data that were previously obtained in Son (2004) and the notion that people may continue studying an item until it has been adequately encoded, in agreement with Toppino et al. (in press). Performance data. The critical question pertains to the performance results. That is, does the spacing effect obtain for items that are both honored and dishonored? The top panel of Figure 2 presents the data comparing honored and dishonored items collapsed across all items. As can be seen, there appears to be a classic spacing effect for items that were honored [t(29) = 2.03, SE = 3.21]. The mean level of performance for items that were massed was 27.48, while that of spaced items was 34.02. However, the results showed no significant improvement for items that were forcibly spaced, against people's choices to mass: The mean performance for those items in which a spacing schedule was imposed was still low, at 28.9 (compared with a mean of 27.48 for the massed items). There was also no difference between performance for massed items--either chosen or imposed (M = 27.61). An additional question can then be raised: At what levels of JOL does the spacing effect hold? In line with the expanding retrieval paradigm results as well as people's choices in general, for example, one might expect that the a spacing effect would not appear if spacing strategies were forced at the very low JOL levels, if those items were actually not fully encoded 4 . The overall ANOVA resulted in increased performance as JOLs increased [F(3, 1879) = 59.37, MSE = 8.76, p = 0, hp 2 = .087], and better 4 Although it is difficult to say whether this is actually so because an item might be given a low JOL because it is either not fully encoded or perceived, or because it has been fully perceived but is simply thought to be difficult to learn. Control and Spacing 13 performance for spaced items than for massed items [F(1, 1879) = 12.40, MSE = 1.83, p = 0, hp 2 = .007]. There were also significant interactions between JOL level and honor/dishonor [F(3, 1879) = 2.63, MSE = .388, p = .049, hp 2 = .004] and between massing/spacing and honor/dishonor [F(1, 1879) = 10.34, MSE = 1.53, p = .01, hp 2 = .005]. There was, however, no significant 3-way interaction. In sum, then, although there were no systematic differences in spacing benefits across JOLs levels, the data show that the spacing effect was not robust enough to obtain across all levels of judged difficulty. Discussion. In summary, the spacing effect was obtained, but only for honored items. When strategy choices were dishonored, the spacing effect disappeared. The short implication here is that deliberately imposing a ―good‖ strategy on the learner, even one that has had overwhelming evidence and agreement in the laboratory, should be applied with caution, particularly when it has not been requested by the learner. The data also imply that there are specific reasons for why an individual would should and would choose a spacing strategy over massing, and those reasons are only perfectly known to the learner himself. Experiment 1B Experiment 1A found a disappearance of the spacing effect when a spacing strategy was forced against one’s own choice. This result indicates that spacing is not an unconditional benefactor to learning. And, at least for the adults tested above, one's metacognitive decisions might in fact characterize the most optimal study schedule for the learner. This is reasonable given that adults are considered to be "fairly experienced 5 " 5 Adult learners have been shown to be far from perfect (e.g. Metcalfe, 1998; Roediger, 1996). Control and Spacing 14 learners in the sense that they are likely to have seen the consequences of massing and spacing in real-world learning, and may be more likely know what is best for themselves. What about a population whose metacognitive decisions might not be as experienced or optimal, as in the case of a child? Will a child’s own decisions be the most optimal for that child? Previous data suggest not. The data demonstrated that very young children (in 1st grade) did not use their metacognitive knowledge to guide spacing choices (Son, 2005), and to boot, preferred massing to spacing practices, at all JOL levels. It seems rational then to believe that imposing a spacing strategy for children might actually enhance learning, in the least at some JOL levels. The method used in Experiment 1B was the same as was used in Experiment 1A, except that the participants were forty-two children in grades 3-5 rather than college students and the materials were age-appropriate for their learning level. In other words, the pairs consisted of elementary-school-aged vocabulary (e.g. occupation job) and the distractor task involved solving 1-digit by 1-digit multiplication problems. The children were also read aloud the instructions and emphasis was put on the fact that they should be as honest in their choices as possible. Finally, a critical difference was that there was no "done" option, since children at this age would be likely to cease any further study on most of the items due to lack of motivation 6 . The children were recruited from a local public elementary school where parental consent for participating was obtained.